虽然微调预训练的网络已成为训练图像分割模型的流行方式，但这种用于图像分割的骨干网络经常使用图像分类源数据集（例如ImageNet）进行预训练。尽管图像分类数据集可以为骨干网络提供丰富的视觉特征和歧视能力，但它们无法以端到端的方式完全预训练目标模型（即骨干+分割模块）。由于分类数据集中缺乏分割标签，因此在微调过程中进行分割模块在微调过程中随机初始化。在我们的工作中，我们提出了一种利用伪语义分割标签（PSSL）的方法，以启用基于分类数据集的图像分割模型的端到端预训练。 PSSL的启发是受到观察的启发，即通过CAM，Smoothgrad和Lime等解释算法获得的分类模型的解释结果将接近视觉对象的像素簇。具体而言，通过解释分类结果并汇总了从多个分类器查询的解释集合来降低单个模型引起的偏差，从而为每个图像获得PSSL。使用PSSL，对于ImageNet的每个图像，提出的方法都利用加权分割学习程序来预先培训分割网络。实验结果表明，在Imagenet伴随PSSL作为源数据集的情况下，提出的端到端预训练策略成功地增强了各种分割模型的性能，即PSPNET-RESNET50，DEEPLABV3-RESNET50和OCRNET-HRNET-HRNETENET-HRNETENET-HRNETENET-HRNETENET-HRNETW18，和在许多细分任务上，例如CAMVID，VOC-A，VOC-C，ADE20K和CityScapes，并有重大改进。源代码可在https://github.com/paddlepaddle/paddleseg上使用。

离线模仿学习（IL）是从没有奖励标签的专家演示中解决决策问题的强大方法。由于协变量转移，现有的离线IL方法在有限的专家数据下遭受严重的性能变性。但是，包括学习的动力学模型可以潜在地改善专家数据的状态行动空间覆盖范围，但是，它也面临着诸如模型近似/概括/概括性错误和推出数据的次级优势之类的挑战性问题。在本文中，我们提出了基于歧视者指导的基于模型的离线模仿学习（DMIL）框架，该框架引入了一个歧视者，以同时区分模型推出数据的动力学正确性和次优性与真实专家示范。 DMIL采用了一种新颖的合作对抗学习策略，该策略使用歧视者指导和融合了政策和动态模型的学习过程，从而改善了模型性能和鲁棒性。当演示包含大量次优数据时，我们的框架也可以扩展到案例。实验结果表明，与小型数据集下的最新离线IL方法相比，DMIL及其扩展具有出色的性能和鲁棒性。

在没有高保真模拟环境的情况下，学习有效的加强学习（RL）政策可以解决现实世界中的复杂任务。在大多数情况下，我们只有具有简化动力学的不完善的模拟器，这不可避免地导致RL策略学习中的SIM到巨大差距。最近出现的离线RL领域为直接从预先收集的历史数据中学习政策提供了另一种可能性。但是，为了达到合理的性能，现有的离线RL算法需要不切实际的离线数据，并具有足够的州行动空间覆盖范围进行培训。这提出了一个新问题：是否有可能通过在线RL中的不完美模拟器中的离线RL中的有限数据中的学习结合到无限制的探索，以解决两种方法的缺点？在这项研究中，我们提出了动态感知的混合离线和对线增强学习（H2O）框架，以为这个问题提供肯定的答案。 H2O引入了动态感知的政策评估方案，该方案可以自适应地惩罚Q函数在模拟的状态行动对上具有较大的动态差距，同时也允许从固定的现实世界数据集中学习。通过广泛的模拟和现实世界任务以及理论分析，我们证明了H2O与其他跨域在线和离线RL算法相对于其他跨域的表现。 H2O提供了全新的脱机脱机RL范式，该范式可能会阐明未来的RL算法设计，以解决实用的现实世界任务。

最近，检测变压器（DETR）是一种端到端对象检测管道，已达到有希望的性能。但是，它需要大规模标记的数据，并遭受域移位，尤其是当目标域中没有标记的数据时。为了解决这个问题，我们根据平均教师框架MTTRANS提出了一个端到端的跨域检测变压器，该变压器可以通过伪标签充分利用对象检测训练中未标记的目标域数据和在域之间的传输知识中的传输知识。我们进一步提出了综合的多级特征对齐方式，以改善由平均教师框架生成的伪标签，利用跨尺度的自我注意事项机制在可变形的DETR中。图像和对象特征在本地，全局和实例级别与基于域查询的特征对齐（DQFA），基于BI级的基于图形的原型对齐（BGPA）和Wine-Wise图像特征对齐（TIFA）对齐。另一方面，未标记的目标域数据伪标记，可用于平均教师框架的对象检测训练，可以导致更好的特征提取和对齐。因此，可以根据变压器的架构对迭代和相互优化的平均教师框架和全面的多层次特征对齐。广泛的实验表明，我们提出的方法在三个领域适应方案中实现了最先进的性能，尤其是SIM10K到CityScapes方案的结果，从52.6地图提高到57.9地图。代码将发布。

Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the prediction capacity. However, there are several severe issues with Transformer that prevent it from being directly applicable to LSTF, including quadratic time complexity, high memory usage, and inherent limitation of the encoder-decoder architecture. To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a ProbSparse self-attention mechanism, which achieves O(L log L) in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences. (iii) the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference speed of long-sequence predictions. Extensive experiments on four large-scale datasets demonstrate that Informer significantly outperforms existing methods and provides a new solution to the LSTF problem.

Sparse principal component analysis (SPCA) has been widely used for dimensionality reduction and feature extraction in high-dimensional data analysis. Despite there are many methodological and theoretical developments in the past two decades, the theoretical guarantees of the popular SPCA algorithm proposed by Zou, Hastie & Tibshirani (2006) based on the elastic net are still unknown. We aim to close this important theoretical gap in this paper. We first revisit the SPCA algorithm of Zou et al. (2006) and present our implementation. Also, we study a computationally more efficient variant of the SPCA algorithm in Zou et al. (2006) that can be considered as the limiting case of SPCA. We provide the guarantees of convergence to a stationary point for both algorithms. We prove that, under a sparse spiked covariance model, both algorithms can recover the principal subspace consistently under mild regularity conditions. We show that their estimation error bounds match the best available bounds of existing works or the minimax rates up to some logarithmic factors. Moreover, we demonstrate the numerical performance of both algorithms in simulation studies.

While deep learning succeeds in a wide range of tasks, it highly depends on the massive collection of annotated data which is expensive and time-consuming. To lower the cost of data annotation, active learning has been proposed to interactively query an oracle to annotate a small proportion of informative samples in an unlabeled dataset. Inspired by the fact that the samples with higher loss are usually more informative to the model than the samples with lower loss, in this paper we present a novel deep active learning approach that queries the oracle for data annotation when the unlabeled sample is believed to incorporate high loss. The core of our approach is a measurement Temporal Output Discrepancy (TOD) that estimates the sample loss by evaluating the discrepancy of outputs given by models at different optimization steps. Our theoretical investigation shows that TOD lower-bounds the accumulated sample loss thus it can be used to select informative unlabeled samples. On basis of TOD, we further develop an effective unlabeled data sampling strategy as well as an unsupervised learning criterion for active learning. Due to the simplicity of TOD, our methods are efficient, flexible, and task-agnostic. Extensive experimental results demonstrate that our approach achieves superior performances than the state-of-the-art active learning methods on image classification and semantic segmentation tasks. In addition, we show that TOD can be utilized to select the best model of potentially the highest testing accuracy from a pool of candidate models.

While mislabeled or ambiguously-labeled samples in the training set could negatively affect the performance of deep models, diagnosing the dataset and identifying mislabeled samples helps to improve the generalization power. Training dynamics, i.e., the traces left by iterations of optimization algorithms, have recently been proved to be effective to localize mislabeled samples with hand-crafted features. In this paper, beyond manually designed features, we introduce a novel learning-based solution, leveraging a noise detector, instanced by an LSTM network, which learns to predict whether a sample was mislabeled using the raw training dynamics as input. Specifically, the proposed method trains the noise detector in a supervised manner using the dataset with synthesized label noises and can adapt to various datasets (either naturally or synthesized label-noised) without retraining. We conduct extensive experiments to evaluate the proposed method. We train the noise detector based on the synthesized label-noised CIFAR dataset and test such noise detector on Tiny ImageNet, CUB-200, Caltech-256, WebVision and Clothing1M. Results show that the proposed method precisely detects mislabeled samples on various datasets without further adaptation, and outperforms state-of-the-art methods. Besides, more experiments demonstrate that the mislabel identification can guide a label correction, namely data debugging, providing orthogonal improvements of algorithm-centric state-of-the-art techniques from the data aspect.

Different from the general visual classification, some classification tasks are more challenging as they need the professional categories of the images. In the paper, we call them expert-level classification. Previous fine-grained vision classification (FGVC) has made many efforts on some of its specific sub-tasks. However, they are difficult to expand to the general cases which rely on the comprehensive analysis of part-global correlation and the hierarchical features interaction. In this paper, we propose Expert Network (ExpNet) to address the unique challenges of expert-level classification through a unified network. In ExpNet, we hierarchically decouple the part and context features and individually process them using a novel attentive mechanism, called Gaze-Shift. In each stage, Gaze-Shift produces a focal-part feature for the subsequent abstraction and memorizes a context-related embedding. Then we fuse the final focal embedding with all memorized context-related embedding to make the prediction. Such an architecture realizes the dual-track processing of partial and global information and hierarchical feature interactions. We conduct the experiments over three representative expert-level classification tasks: FGVC, disease classification, and artwork attributes classification. In these experiments, superior performance of our ExpNet is observed comparing to the state-of-the-arts in a wide range of fields, indicating the effectiveness and generalization of our ExpNet. The code will be made publicly available.

Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this paper, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose.